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ABSTRACT 

A simulator facility employing a two-axis compensatory tracking 
task with a random-appearing signal was used to evaluate the perform- 
ance of fifty-five pilot and non-pilot test subjects using four separate 
control sticks — two movable and two rigid. Pilot acceptance of the 
rigid cockpit controllers was determined by comparing individual pilot 
ratings of the sticks. In general, in both performance and opinion, the 
rigid systems were found to be superior to their movable counterparts. 
Steps were taken to avoid errors due to pilot bias, learning, adaptation, 
or fatigue. The results obtained are subject to several test limitations, 
including the low stick-force levels used, the neglect of aircraft 
vibration effects, and the realism of the simulation. 
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I. INTRODUCTION 



With the evolution of modern-day high-performance aircraft, 
more reliable and responsive flight control systems will be required. 

As the simple mechanical control systems of the past are replaced by 
complex linkages and fully-powered and power-boost controls, numerous 
problems concerned with flight control weight, nonlinearities, friction, 
hysteresis, inertia, and backlash arise. These problems, together with 
reliance on stability augmentation systems to assist the pilot in portions 
of the flight envelope, have stimulated investigations of fly-by -wire 
techniques (electronic control systems). Fly-by-wire research has now 
advanced to the stage where test flights demonstrating reliability are 
being made and are reporting favorable results (Ref. 2). 

With a basic change from mechanical to electrical control systems, 
various possibilities exist for a cockpit controller different from the 
conventional deck-mounted movable stick. Numerous manipulators have 
been studied, the most prominent of which is a side-located, limited- 
motion hand controller (Refs. 3, 4, 5, 6, 7). These systems allow increased 
cockpit space for flight displays or for additional control functions and 
depend, for the most part, on a reliable fly-by-wire capability. 

The most important considerations, when experimenting with control 
sticks, are the aircraft handling qualities. The pilot regards stick feel 
as a particularly valuable cue in maneuvering the vehicle (Ref. 8). 

With the irreversibility of power-assisted electronic flight control 
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systems, this feel must be provided artificially to the stick — whether 
centered or side-located. Since the pilot relies heavily on this stick 
force, the actual motion of the controls is of much less importance. 

In fact it is widely recognized that a pilot seldom, if ever, knows what 
position his control stick is in (Refs. 5 , 9). This suggests a force-only 
or rigid controller could be applicable to a fly -by-wire control system. 

A rigid stick might prove more satisfactory, if not for primary control, 
for a back-up precision tracker to be used for formation flying, terrain 
avoidance, gunnery runs, weapon control, carrier landings, or ground 
control approaches. Limited work has been done with rigid sticks 
(Refs. 5, 10, 11, 12, 13, 14, 15), and the results have been contra- 
dictory and essentially inconclusive. 

Since handling qualities are inevitably determined from pilot 
opinion, a simulator facility was developed by Commander D. W. Caswell 
(Ref. 1) to evaluate pilot acceptance of a rigid cockpit control system 
and test their ability using it. The simulator employs a two-axis, 
compensatory tracking task with a repeatable random-appearing signal. 
This investigation used this simulator to measure pilot performance for 
fifty-five test subjects on each of four control sticks — two movable 
and two rigid. This performance was compared with individual pilot 
ratings of the separate controllers. Personal pilot experience data were 
collected to insure a thorough test subject analysis. Scores were also 
recorded during a portion of the test runs to determine the effect on the 
data, if any, of pilot learning, adaptation, or fatigue. An approximate 
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human transfer study was conducted using two sticks and two subjects 



for the purposes of correlation. Additional qualitative comments from 
the subjects were recorded and a statistical score-to-rating correlation 
study was made. 
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II. THE SIMULATOR FACILITY 



The simulator (shown in Fig. 1) for pilot evaluation of the rigid 
and movable control sticks presents a two-dimensional tracking problem 
to the test subject. An Ampex tape recorder provides a repeatable test 
signal which is displayed on an X-Y cathode ray oscilloscope with a 
five-inch grid. The "instrument panel" is shown in Fig. 2. The pilot- 
operated control stick generates a signal which, when amplified, acts 
to cancel out the random input signal from the tape — moving a pip to 
the center of the grid. This is accomplished with a summing amplifier. 

The control stick signal is altered by an analog computer circuit to simu- 
late actual aircraft dynamics. Thus, in his efforts to center the pip on 
the oscilloscope screen, the test subject has a constant display of his 
error on the grid. A Brush recorder display of the test signal and a typical 
pilot’s error function in the lateral mode is shown in Fig. 3. 

A. THE CONTROL STICKS 

Four separate controllers were used in the test program: a movable 
and rigid deck-mounted (centered) stick, and a movable and rigid side- 
located manipulator. The movable sticks employed variable potentio- 
meters for signal generation, while the rigid sticks used strain gages in 
a Wheatstone bridge circuit. A light stick force was provided in the 
movable controllers by artificial-feel springs, although in the movable 
side-arm system the spring force was inadequate to center the stick. 
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Thus, the side-located controller was essentially free-moving or 
unrestrained . 

In the rigid sticks, allowing negligible deflections, the control 
force bending moments activated the strain gages providing the control 
signal. The rigid controllers were constructed identical in size to their 
movable counterparts, and their force levels were very low to provide 
a similar control action for each stick. 

All the sticks were wired to a sixteen-connector plug for a quick- 
change capability. The control signals were amplified by a factor of 
two hundred for the rigid and one for the movable systems before their 
input to the analog computer dynamics circuits. 

B. COMPUTER CONTROL DYNAMICS 

The lateral and longitudinal control signals were changed by the 
analog computer circuits to simulate the response of an aircraft. The 
longitudinal circuit provides a damped short -period oscillation approxi- 
mating the dynamics of the F-4 aircraft at a Mach number of 0.9 at sea 
level. The computer response to a step input in pitch is shown in Fig. 4. 
The lateral circuit provides a damped first-order response simulating 
a stable aircraft. A step input in yaw gives the response shown in Fig. 5. 

C. THE TEST SIGNAL 

The random input signal from the tape deck consisted of eight 
segments: 
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1) Two minutes zero signal (pip centered) 

2) One minute longitudinal signal only 

3) Thirty seconds zero signal 

4) One minute lateral signal only 

5) Thirty seconds zero signal 

6) One minute combined longitudinal and lateral signal 

7) Thirty seconds zero signal 

8) Three minutes combined signal (scoring run). 

A Brush recording of the entire scoring run showing the signal magnitudes 
in both the lateral and longitudinal channels is shown in Fig. 6. 

D. THE SCORING MECHANISM 

The scoring of a test subject is based on the time duration (in 
seconds) of errors which drop below a pre-set level, i.e. , the amount 
of time the pip is in a small scoring circle in the center of the oscillo- 
scope grid. When the error exceeds the specified amount (when the pip 
leaves the circle), a signal comparator stops the oscillator signal which 
runs the timer. A green indicator light on the instrument panel also goes 
out, telling the pilot he is out of the scoring circle. The panel also con- 
tains a blue indicator light to signal when the scoring run is in progress. 

A complete list of equipment used is given in Appendix A. Further, 
more detailed information concerning the simulator facility may be found 
in Ref. 1 . 
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Figure 1. The Simulator Facility 










Figure 2. 



The Cockpit Instrument Panel 
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Figure 3. Typical Pilot Tracking Error 
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Figure 4. Longitudinal Response to Step Input 





Figure 5. Lateral Response to Step Input 
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III. PROCEDURES FOR TESTING AND PILOT EVALUATION 



Before the beginning of each test run, the subjects were briefed 
on the operation of the simulator and the testing plan. Explanations 
were given regarding the nature of the various control sticks, the size 
of the scoring circle, and the control motion and force to give a desired 
pip deflection. Also the meaning of the blue and green indicator lights, 
the tape input signal order, the testing order of the sticks, and the 
length of the run were given. 

The test was initiated by selecting the proper stick switch on the 
patch panel and turning on the tape. The first two-minute segment of 
the tape input — the zero signal — was used to b^Uance the potentio- 
meters (centering the scoring circle on the grid) and to allow the pilot 
to center the pip. The subject then used the one minute of longitudinal 
signal, one minute of lateral signal, and one minute of the combined 
two-axis signal with which to practice and to become adapted to the 
particular control stick. The final three-minute segment of the input 
signal, the two-degree-of-freedom motion, was used for the scoring run. 
The pilot was informed when the test signal changed modes and was 
allowed thirty-second rest periods between the various segments of the 
tape. Adequate warning was given prior to the beginning of the scoring 
run, and the subject was also notified when one minute remained in the 
test. At the end of the run, the tape was rewound, a changeover switch 
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was made to another stick , and the testing process repeated using the 



identical input signal. 

At the end of the test, the subjects were asked to complete a 
questionnaire evaluating the control sticks and providing precise informa- 
tion concerning their total flight experience. These evaluations were 
made before the pilots were told their scores on each stick. The question- 
naire used is shown in Table I. 

For an adequate pilot evaluation of the controllers, a rating scale 
was required. In the past, handling-qualities research has usually 
employed the standardized Cooper rating scale (Ref. 16) or, more recently, 
the modified Cooper-Harper scale (Ref. 17). Caswell's preliminary 
testing (Ref. 1) used the original Cooper scale and several of his testees 
gave two different sticks an identical rating. In an effort to avoid this 
situation, a new rating scale was devised to allow for a finer discrimi- 
nation in the evaluation of the four control sticks. This scale allows a 
wider range of "satisfactory" ratings, while still providing the oppor- 
tunity for giving an adverse opinion. The revised rating scale is given 
in Table II. 

Caswell's testing also found a tendency for the test subjects' 
performance to improve as the test progressed from one stick to another. 
For this reason the testing order was varied to cancel out the effects of 
a possible learning function. Since the primary purpose of this study was 
to compare rigid with movable controls rather than center with side- 
located sticks, the variation of test order was concerned only with 
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alternating the movable versus the rigid systems. Thus in each system 
the deck-mounted center stick was used first. 

To determine any possible learning, fatigue, or adaptation during 
the course of one run, partial scores were observed and recorded at 
thirty-second intervals for thirteen test subjects. Finally, qualitative 
comments by the pilots concerning the validity of the simulation and the 
applicability of a rigid cockpit control system were recorded. 
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TABLE I 



PILOT QUESTIONNAIRE 

Name Test Number 

Age Date 

FLIGHT EXPERIENCE 
Approximate Number of Pilot Hours in: 

Single-Engine Jet 

Multi-Engine Jet 

Single-Engine Prop 

Multi-Engine Prop 

Helicopter 

Light General A/C 

Non-Pilot 

Type of Operational A/C with Most Experience (F-4, A-4, etc.)? 
Type of Operational A/C with Most Recent Experience? 

How Long Since Piloted Any Aircraft ? (If over one month) 

How Long Since on Full-Time Operational Flight Status ? 



PILOT OPINION AND PERFORMANCE 

Pilot Rating Test Score Preference 

Movable Center Stick 

Movable Side-Arm Stick 

Rigid Center Stick 

Rigid Side-Arm Stick 
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TABLE II 



PILOT RATING SCALE 
Numerical Rating — Descriptive Phrases 

1 - Fantastic, could not be improved, should be in all aircraft. 

2 - Excellent control response , no gripes . 

3 - Good response, pleasant to fly. 

4 - Good response, but would require some getting used to. 

5 - Satisfactory operational response, would expect no difficulty. 

6 - Satisfactory, would expect minor problems in certain situations. 

7 - Acceptable, but with some unpleasant characteristics. 

8 - Unacceptable for normal operations . 

9 - Unacceptable for any operations . 

10 - Unsatisfactory, dangerous, uncontrollable. 
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IV. ANALYSIS AND CLASSIFICATION OF TEST SUBTECTS 

To properly interpret the data in a test of this sort, it is necessary 
to have background information on the test subjects. The extensive 
flight experience data obtained from the pilot questionnaire is given in 
Appendix B. 

Included in the fifty-five subjects tested were Naval aviators, 
Naval Flight Officers, private pilots, and non-pilots representing a 
broad spectrum of aircraft flight experience. The private and non-pilots 
tested can be considered as representative of the type of personnel who 
enter Naval flight training. 

A. PILOT CLASSIFICATION 

To facilitate an interesting and meaningful evaluation of the scores 
and ratings, the testees were classified into five major groups; 



1) Jet pilots 18 

2) Propeller (prop) pilots .... 14 

3) Helicopter (helo) pilots... 9 

4) Private (pri) pilots 8 

5) Non-pilots 9 



It is seen that a relatively even distribution was obtained — there are 
enough subjects in each category to insure an accurate data base. Three 
pilots (Numbers 5, 12, and 29) had significant experience in two differ- 
ent categories and thus were included in both. The classification of each 
pilot is also included in Appendix B. 
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This division was felt necessary for several reasons. First, it 



was thought necessary to evaluate the opinions and performance of the 
pilots in light of their experience. Different types of pilots use differ- 
ent control sticks. In general, the jet and helicopter pilots are experi- 
enced with the conventional, deck-mounted movable stick, while 
propeller and private pilots use a yoke or wheel controller. Single 
engine propeller aircraft do have the movable center stick, however 
only two pilots had the majority of their experience with this type 
(flight instructors). Second, pilot classification of the data could give 
an indication of the type of aircraft in which a rigid stick might be more 
applicable. For example, if jet pilots should find a particular prefer- 
ence for the rigid control stick, this system might be of use in jet air- 
craft. Finally, the performance and evaluation by non-pilots compared 
with those of pilots is necessary to determine the effect of long- 
established flying tradition on the opinions of the pilots. It must be 
mentioned, however, that the non-pilot group tested did have a limited 
amount of flight experience, although in most cases, it was less than 
twenty hours . 

B. TEST SUBJECT ANALYSIS 

The age of the average subject tested was 27.9 years, with 
thirty-seven of the testees between 27 and 32 years of age. A signifi- 
cant age difference of 5 to 12 years was noted between the non-pilots 
and pilots. 
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Thirty-seven of the forty-six pilots had flown some type of air- 
craft within one month of the day they were tested (proficiency flying) . 
The average time since any flight was 2.77 months, including only five 
subjects who had gone over a year without flying. In general, the private 
pilots had not flown recently. 

The thirty-eight Navy and Marine pilots tested were students and 
not on operational flight status. The average time since they had been 
flying operationally was 13 months. Twenty-seven pilots had been on 
operational status between 5 and 12 months prior to their test. Only 
five of the Navy pilots had not been on flight status in more than 2 ^ 
years . 

The airplanes in which the thirty-eight military pilots had the most 
experience present a wide variety of Naval aircraft. These are: 

JET: A-4, A-6, A-7, F-4 , F-8 

PROP: A-1, P-5, P-2, P-3, S-2, T-28, C-121 

HELO: H-2, H-4, H-34, SH-3 

Twenty-nine of the Navy and Marine pilots had their most recent experi- 
ence in the same aircraft in which they had the most experience. Only 
one pilot had his most recent experience in an airplane of a different 
type (a helo pilot in an S-2). 

The average military pilot tested had flown 2200 hours total flight 
time, with thirty-three out of thirty-eight pilots between 1000 and 4000 
hours. The private pilots had significantly less flight experience, with 
an average of 200 hours apiece. 



26 



I ii 




Thus in general it can be seen that the subjects tested were 



moderately experienced military pilots, about 30 years of age, who had 
been on operational flight status one year ago and were making proficiency 
flights in their aircraft type at the time of their test in the simulator. 
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V. TEST RESULTS 



By the end of the testing program, results were obtained from 
fifty-five subjects, including nine non-pilots. The data obtained — test 
scores, pilot ratings, and testing order — are given in Appendix C. 
Several methods of data reduction and comparison were utilized for a 
complete and meaningful evaluation of the test results. Average scores 
and ratings were computed for each pilot classification and for the 
entire test group. Since averages can give an incomplete view, the 
raw scoring and pilot rating distributions are given to supplement the 
averaged performances and opinions. In addition to the comparison of 
the various control sticks, the individual pilot classifications are 
compared to further illuminate the subject. 

A. AVERAGES 

The over-all average test score for each stick, out of a possible 
total of 180 seconds, is given in Fig. 7. It is evident that the body of 
test subjects did better with the rigid controllers than with the movable 
sticks. The best performance was on the rigid side-arm stick, while 
the poorest by far was with the movable hand controller. The average 
score with the movable center stick, while lower, compared favorably 
with the rigid systems. 

Figure 8 presents the average pilot rating for each stick. The 
rigid sticks were preferred overwhelmingly over the movable controls. 
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The force-only sticks were found to have "good response," while the 
movable controllers were rated only "satisfactory." The subjects tended 
to dislike the movable side-arm stick, and little difference was noted 
between the two rigid manipulators. At least on the average, pilot 
opinion correlated with actual performance. A more detailed score-to- 
rating correlation study is included in Section VI. 

B. COMPARISON OF STICKS 

More detailed comparisons of the four control sticks for a parti- 
cular pilot class are shown in Figs. 9 and 10. These results (Fig. 9) 
show that all groups do better with the rigid controls to varying degrees. 
We also see that jet, helicopter, and non-pilots exhibit performance 
variations identical to those of the over-all averages (Fig. 7); i.e. , 
rigid side-arm — best and movable side-arm — worst. Notable differ- 
ences are : 1) the prop pilots do slightly better with the rigid center 
stick than with the rigid hand controller, rather than vice versa, and 
2) private pilots scored higher with the movable side stick than with 
the movable deck-mounted stick. 

It is evident from Fig. 10 that all pilot groups preferred the rigid 
to the movable controls and disliked the movable side stick. Opinion 
was fairly evenly divided as to which stick was the best — rigid center 
or rigid side-arm. The relative magnitudes of all the rating averages 
are about the same, indicating the pilots' interpretations of the rating 
scale were quite uniform. 
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C. COMPARISON OF PILOT CLASSES 

Actual comparison of the different pilot groups for a particular 
control stick are given in Figs. 11 and 12. This analysis reveals that 
private pilots and non-pilots scored higher than pilots with the rigid 
control rticks. As expected, jet pilots performed superior to the other 
groups using the conventional movable center stick. The results also 
indicate prop pilots encounter difficulty with the side sticks, and both 
prop and helo pilots score generally below average on all sticks. 

D. SCORE AND RATING DISTRIBUTION 

To contribute to the averaging analysis, charts of the test score 
and pilot rating distributions were made. The over-all performance 
distribution plot of Fig. 13 gives the percentage of the total number of 
test subjects whose final score on a particular stick was in each 5-second 
time interval throughout the range of test scores. A statistical distribu- 
tion of scores about the average is approached with the two rigid sticks 
and the movable center stick (Fig. 13a, b, c). However, the movable 
side-arm stick is marked by a rather even distribution of scores concen- 
trated at the lower end of the scale. 

The over-all pilot opinions (Fig. 14) are somewhat more divided on 
the movable sticks than on the rigid systems — especially in the case of 
the movable hand controller, where the ratings are particularly scattered. 

Identical performance and pilot opinion distribution charts for the 
different pilot groups are included in Appendix D. Facts of interest from 
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these graphs are; 1) Jet pilot scores on the movable side stick were 
divided in two groups — one average and one poor (Fig. D-ld), 2) Two 
individual prop pilots scored extremely low on the rigid hand stick — lower- 
ing the over-all performance average for the group (Fig. D-3b) , and 3) 

While the scores of private pilots were concentrated, their opinions of 
all sticks were scattered — indicating not enough private pilots were 
tested fora valid comparison (Figs. D-7, D-8). 
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Figure 9. Average Test Score by Each Pilot Classification 
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Figure 10. Average Rating By Each Pilot Classification 
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Figure 11. Average Test Score By Pilot Class For Each Stick 
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Figure 12. Average Rating By Pilot Class For Each Stick 
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VI. HUMAN FACTORS INVOLVEMENT \VITH TEST VALIDITY 



In a test program of this nature, dealing with variations in pilot 
ability and the vagaries of personal opinion, a great deal of care must 
be taken to insure validity of the data. Human involvement errors can 
be introduced by myriad differences in adaptation, learning, fatigue, 
and experience, and by fluctuations in attention, motivation, and judge- 
ment. Safeguards must be established and checks need to be made to 
cancel these possible sources of error. 

To determine the nature of human involvement, several methods 

) 

of analysis were utilized. A score -to-rating correlation study was made 
to determine the effects, if any, of pilot bias. A standard regression 
analysis was applied to the data as a means of representing each score 
and rating on a single graph. Learning, adaptation, and fatigue of the 
test subjects were explored by observing the variation of performance 
throughout a run and then throughout the test — the testing order of the 
four sticks being a key parameter. Additional information concerning 
learning and adaptation as well as variation in stick dynamics was 
sought by an approximate human transfer function study with the simu- 

• / . I 

lator. A check was also made to find the variation in test results due 

I 

to a change in the magnitude of the pip deflection, since it could possibly 
have been altered at some time during the five-month testing program. 
Finally, the realism or validity of the simulation was studied by comparing 
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non-pilot performance to that of pilots and by soliciting individual 
comments from the pilots tested. 

A. SCORE-TO-RATING CORRELATION 

With the introduction of a different new device like the rigid 
control stick, the possibility exists that a pilot could become enhanced 
with its novelty and evaluate it accordingly. On the other hand, a pilot 
with many hours experience with a movable deck-mounted stick might 
tend to dislike the rigid controls, even though his performance is superior 
using them. To determine the magnitude of this type of problem in the 
testing program, a score-to-opinion correlation analysis was applied. 

I 

Plotting the over-all average scores versus the average ratings for each 
stick as shown in Fig. 15 gives a first approximation as to how well 
the pilot opinions correlate with performance. The strong correlation is 
obvious, since the rating becomes rapidly less favorable (higher) with 
a decreasing test score. Similarly, in Fig. 16 a regression analysis 
plotting score versus rating and reducing the points to a single line for 
each stick shows this strong correlation of performance to pilot 
evaluation. 

Fora more detailed analysis, an individual correlation factor "r" 
was calculated for each pilot. These factors are listed in Table III, with 
a perfect correlation being r = -1, since the pilot rating decreases as 
the performance increases. From the table we see that twenty-nine of 
the subjects have a high correlation of -.8 or better. Fourteen (25%) of 
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of the subjects tested have a poor score-to-rating correlation (above 
-.4). A close examination of the scores and ratings of these fourteen 
testees reveals the causes of the poor correlations. Eight of these sub- 
jects — Numbers 13 , 22 , 35 , 39 , 40 , 43 , 50 , and 52 — were excluded from 
the correlation analysis, because their poor correlations were due to 
nearly identical scores on separate sticks or to identical ratings to more 
than one stick. The remaining six poor correlations were possibly influ- 
enced by pilot bias. Four of these subjects preferred the movable con- 
trols but scored higher with the rigid systems. The remaining two 
subjects scored well with the movable sticks but preferred the rigid 
controls . 

The average correlations for each pilot class are of interest and 
are given in Fig. 17. As expected, the less -experienced private and 
non-pilots had significantly lower score -to -rating correlations than 
those of the military pilots. 

Thus, with a high over-all average correlation of - .68, it appears 
that pilot bias had little effect on the data. The majority of the subjects 
had relatively high performance-to-opinion correlations. The poor cor- 
relations were small in number, and the errors induced tended to cancel 
one another. 

B. LEARNING, ADAPTAHON, AND FATIGUE 

In the preliminary testing by Commander Caswell (Ref. 1), the 
pilots' skill with the simulator tended to improve as the test progressed. 
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To avoid allowing one type of stick to obtain an advantage over the 
other, the testing order (movable vs. rigid) was changed for each sub- 
ject. Thirty subjects used the rigid controls first, and the remaining 
twenty-five started with the movable sticks. Thus any possible learning 
function effects should have been nearly eliminated. Evaluating the 
increase in score from the first stick used to the next, the average 
learning was of the order of one second and thus essentially negligible. 

An almost universal comment from the pilots upon finishing the 
test was that the tracking task was extremely tiring. To further reveal 
learning, adaptive, or fatigue factors, partial scores were recorded at 
30-second intervals throughout the three-minute test run for fourteen 
test subjects. The results obtained are displayed in Fig. 18. It is 
evident thaftest scores dropped off rather rapidly in the final 30-second 
frame. This could be due to fatigue or possibly due to the fact that the 
pilots were given a notification one minute before the end of a run. 
Motivation may have lowered slightly in anticipation of the end of a 
somewhat tedious test. 

Little evidence of slow adaptation to the sticks can be found on 
the graph. The subjects in general appear to have become fully adapted 
to each controller during the practice time preceding the scoring run. 

The scores in the first frames show only a slight improvement in perform- 
ance, with actually a decrease using the movable side-arm stick. This 
is not surprising since human operator control adaptation usually occurs 
in 1-3 seconds following a change in simple tracking conditions (Ref. 18). 



43 



An effort was made to compute an approximate human transfer 
function for two test subjects using the simulator facility. It was hoped 
a further more analytical description of human adaptation and learning 
could be found (Ref. 19). The subjects completed a sinusoidal tracking 
task in the lateral mode at various frequencies of oscillation, using the 
movable center stick and the rigid side-arm controller. Their frequency 
response was recorded and is illustrated in the Bode plots of Figs. 19 
and 20. Pilot transfer functions were obtained from these plots by the 
asymptotic approximation method. The similar form of the curves indi- 
cates that human response, and thus adaptation, is comparable for a 
given stick; but the response to two different sticks can be quite diverse. 
It was not attempted to apply these results to the data since the study 
was approximate, and indication of adaptive difficulty in the tests was 
absent. 

C. SCORING PIP DEFLECTION 

Since the testing program was extended over a several-month 
period, the motion of the pip on the oscilloscope for a given control 
force could quite possibly have been changed. This could have occurred 
due to other use of the Ampex tape recorder or actual tape deterioration. 
To check this source of error, three subjects were tested at two different 
pip deflections — one large and one small. As might be expected, the 
scores with a small deflection were much higher as the pip was easier to 
control. However, little change was found in the relative differences 
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among the four control sticks. Thus it was assumed that even if slight 
changes occurred in the pip deflection, the error, for stick comparison 
purposes, was minimal. 

D. TEST LIMITATIONS AND SIMULATION VALIDITY 

Most pilots seemed to agree that while their scores and ratings 
were higher using the rigid systems in the simulator, the situation might 
be significantly different in an actual aircraft. Herein lies the major 
limitation to a simulated opinion survey of this sort. It was felt by the 
pilots that actual flight testing is necessary to determine, for sure, 
pilot acceptance or rejection of a rigid control stick. The relatively high 
scores of the non-pilot group cast further doubt on the realism of the 
simulation; the age differential between pilots and non-pilots could be 
a factor, however. 

Another key limitation concerns the low force gradients incorpor- 
ated into all four sticks. The maximum force necessary was about one 
pound, while the optimum stick force per g in an actual aircraft is 5-7 
pounds (Ref. 20). A more thorough and comprehensive study would 
include the effects of variations of stick force per pip deflection on 
pilot opinion and performance. Of particular note in this investigation 
was the essentially free-moving, unrestrained motion of the movable 
side-arm controller. Quite possibly a spring-restrained movable hand 
stick would fare much better in comparison with the rigid systems. On 
the other hand, higher force levels in the rigid sticks might make them 
even more acceptable. 
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This survey is further limited in that the relative effects of vibra- 



tion using the control sticks were ignored. Aircraft vibration can reduce 
the manual dexterity of the pilot and introduce an additional unsteadiness 
in his control motion. The human body tends to damp out vibration; thus 
problems could arise with side-stick controllers where the pilot’s arm 
rests on a surface vibrating v/ith the airplane. Reference 2 reported that 
a movable control stick gave performance superior to the rigid controller 
at all exciting frequencies of vibration tested. Vibration would naturally 
enter a flight testing program and could also be introduced into a simu- 
lator. The effects of acceleration on control stick performance must also 
be studied in a flight test rather than a simulator investigation. 

During the course of the testing several pilots were concerned 
about the effect on a rigid controller of conventional stick-mounted trim 
switches and microphone buttons. While these switches create no stick 
motion when in operation, they would involve the addition of an extra 
force on a rigid control stick. This would mean applying an equal and 
opposite force while using trim tabs to avoid an unv/anted control deflec- 
tion. This problem could be included in a further simulator study. 

A final limitation might lie in the fact that the subjects tested 
were not trained test pilots as is the custom in handling-qualities simu- 
lator studies. The limited qualitative nature of information requested in 
the evaluation of the sticks, however, should have relieved this require- 
ment. The high score-to-rating correlations achieved by the pilots 
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supports this assumption. In addition, it was desired to determine 
the opinions of the average fleet pilot with regard to acceptance of 
a rigid cockpit control system. 
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TABLE III 



INDIVIDUAL PILOT SCORE-TO-RATING CORRELATION FACTORS 



Subject 


r 


Subject 


r 


Subject 


r 


1 


-.912 


20 


+ .206 


38 


-.952 


2 


-.978 


21 


-.982 


39 


-.149 


3 


-.718 


22 


-.394 


40 


-.109 


4 


-.972 


23 


-.546 


41 


-.990 


5 


-.350 


24 


-.829 


42 


+ .803 


6 


-.951 


25 


-.971 


43 


0.000 


7 


-.460 


26 


-.642 


44 


-.918 


8 


-.539 


27 


-.932 


45 


-.941 


9 


-.619 


28 


-.952 


46 


-.669 


10 


-.911 


29 


-.994 


47 


-.900 


11 


-.660 


30 


-.985 


48 


-.993 


12 


-.697 


31 


-.965 


49 


-.758 


13 


+ .978 


32 


-.926 


50 


-.007 


14 


-.768 


33 


-.900 


51 


-.914 


15 


-.804 


34 


-.204 


52 


-.260 


16 


-.746 


35 


+ .453 


53 


-.004 


17 


-.948 


36 


+ . 576 


54 


-.532 


18 


-.833 


37 


-.426 


55 


-.876 



19 -.967 



A perfect score-to-rating correlation is r = -1.00. 
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Figure 15. Correlation of Pilot Opinion to Performance 
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Test Score — seconds 
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Figure 16. Score-to-Rating Regression Analysis 
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Figure 17. Average Correlations of Scores to Opinions 



51 



Score in 30-second Time Frame — seconds 
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Figure 18. Average Pilot Scoring Pace Throughout Test Run 
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Subject 2 
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Figure 19. Pilot Frequency Response With Rigid Side-Arm Stick 
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Figure 20. Pilot Frequency Response With Movable Center Stick 



VII. DISCUSSIONS AND CONCLUSIONS 



The purpose of this investigation was to determine the accept- 
ability to operational military pilots of a rigid cockpit control system 
and to evaluate their performance using the force-only controls. Within 
the restricting framework of the test limitations mentioned in the previous 
section, this has effectively been accomplished. Results were obtained 
from fifty-five flying and non -flying test subjects operating both movable 
and rigid controllers in a compensatory tracking task. 

To draw specific and meaningful conclusions from the data collected, 
it is necessary to thoroughly analyze the test subject group in order to 
determine applicability of the results. At the same time, the particular 
measurement standards and procedures must be reviewed to insure validity 
of the test. Finally, in a study involving human opinions, the possible 
sources of error induced by the subjects and inherent in the test itself 
must be examined. 

The test subject group consisted of Navy and Marine aviators. 

Naval Flight Officers, private pilots, and non-pilots. The pilots tested 
were on a proficiency flying status and thus may not have been quite as 
sharp as their active fleet counterparts. The non-pilot group was highly 
representative of the personnel who enter Naval flight training (future 
pilots). A significant number of each pilot classification — jet, propeller, 
helicopter, private, and non- pilot --was tested to form a sound base for 
significant conclusions. The exception was the private pilot class who 
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showed basically erratic performance and ratings, possibly due to the 
duration of time since many of them had flown. In general, however, 
the inexperienced subjects were tested to compare their performance with 
that of pilots --their opinions were of lesser importance in the test. 

The wide variety of aircraft in the pilots' background experience repre- 
sented the majority of operational aircraft in the fleet today. 

The measurement procedures in this investigation were relatively 
conventional and test validity was protected. The signal error in a com- 
pensatory tracking task is a common measure of pilot performance and 
is a fairly standard procedure. The revised rating scale, allowing the 
opportunity for finer discriminations, was devised especially for the 
test. Its success is evidenced by the relatively uniform interpretations 
it received by the various pilot classifications. 

Numerous precise measurements were conducted to eliminate 
errors induced by the human test subjects and test apparatus. The 
extensive correlation analysis effectively ruled out pilot bias in most 
test subjects. It demonstrated that the small amount of bias found tended 
to cancel itself out; i.e. , some pilots preferred the movable system and 
scored high with the rigid, while others preferred the rigid and scored 
well with the movable controls. Alternating the testing order of the 
sticks virtually eliminated pilot learning during the course of the test. 
Even so, the learning value was a very small number. Most of the pilots 
experienced fatigue during the final 30 seconds of a run; however, this 
only introduced a difference of 2-3 seconds in test score. The fatigue 
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was slightly greater with the movable controls --another indication of 
their difficulty, in addition to pilot scores and ratings. No problem was 
experienced with pilot adaptation to a particular control stick. The tape 
segment allotted for practice allowed time for a complete adaptation. 

The error due to a possible change in pip deflection was also negligible, 
since the relative differences of scores and ratings for each stick remained 
essentially constant. Thus various safeguards and checks greatly mini- 
mized these various possible sources of error in the data. 

In general the pilots performed better with the rigid control sticks 
and preferred them over the movable systems. The rigid side-arm control- 
ler was the consensus favorite in both performance and preference. The 
movable side-arm stick was uniformly disliked and its performance was 
inferior. The movable deck-mounted stick was only slightly less preferred 
by the pilots than the rigid systems — the differences in scores and ratings 
were not large; whereas the less experienced private and non-pilot group 
displayed a rather marked preference of rigid over movable controllers 
with the coincident large difference in performance with each system. 
These results suggest that a rigid controller is certainly feasible for 
either a primary control, back-up control, or precision tracker in an air- 
craft with fly-by-wire capability. It can also be concluded that it would 
not be economically feasible to replace present movable controls with 
force-only sticks, since the difference in performance and present-pilot 
rating is not large. Therefore on the basis of the preliminary results of 
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this test, the rigid control systems appear feasible for future primary 



control systems or present secondary tracking functions. 

This acceptability and feasibility of the rigid sticks must be 
examined in light of the test limitations. The foremost consideration 
in a simulator study is the degree of realism of the simulation. Although 
many of the pilots expressed concern over this matter, several indica- 
tions supported an assumption of "airplane-like" simulation: 1) As 

would be expected, the jet pilots scored higher than all other groups 
using the conventional movable center stick, 2) The less experienced 
private and non-pilots had a greater divergence of scores between the 
movable and rigid systems, and 3) Individual pilot correlations were 
higher than those of private and non-pilots, as would be expected of 
men with greater flying experience. 

The major simulation limitations deal with low stick force gradi- 
ents , lack of vibration (as is found in aircraft), and omission of acceler- 
ation effects on performance and opinion. This report should not be 
considered an indictment of a movable side-arm controller. The 
unrestrained nature of this stick badly hampered its ability to compete 
with the other three sticks. Vibration effects were not included, and 
most reports say it detracts from rigid control effectiveness. Acceleration 
could not be included in a simulator study, of course; there is a strong 
possibility that a rigid control system is superior under high g condi- 
tions, due to the difficulty of gross arm motion. 
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Thus all of these limitations could seriously affect the pilot 
performance and acceptance of rigid control systems reported in this 
study. The results here essentially indicate a preliminary feasibility, 
with a strong requirement for future investigation and flight testing. 
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VIII. RECOMMENDATIONS 



Viewing the results of this investigation, several specific 
recommendations concerning future compensatory tracking tests and 
future work with rigid control systems come to mind. 

A. CONTROLLER EVALUATION TESTS 

When evaluating more than one control stick using an identical 
signal input for each one, as was the case in this investigation, a 
great deal of time could be saved by having the four identical signals 
on one long tape rather than having one signal which requires rewinding. 
Since the subjects encountered little or no adaptation troubles, the 
rest periods and the practice runs could be shortened somewhat. These 
measures would tend to shorten the test time for an individual subject 
and thus decrease his fatigue. 

Future control stick evaluation studies should also allow the 
pilots to rate other qualities of the controller such as sensitivity, 
location, feel, deflection, force levels, naturalness of control, good 
and bad features, and coordination of the control modes. This would 
overcome possible limitations of the 10-point rating scale in this test. 
For finer discriminations between two nearly identical sticks (such as 
rigid center vs. rigid side-arm) the pilots could also be allowed to 
give fractional ratings such as 3.5 or 4.6. 
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B. RIGID CONTROL SYSTEMS 

Before a precise judgement can be made as to the acceptability 
and applicability of a rigid controller, several problem areas should 
be investigated. Chief among these is the matter of stick force gradi- 
ents. A man-machine optimization testing program is required to com- 
pare the rigid and movable control systems through a range of stick 
forces. The optimum force and motion combination for a movable stick 
should then be compared to the optimum stick force for a rigid system. 
Only then can the applicability of a rigid system be determined — both 
on the basis of performance and pilot opinion. A system has already 
been designed which allows the test subject to alter his own force 
levels in the cockpit simulator. 

A second major area to be investigated is the effects of aircraft 
vibration on a pilot's performance and opinion. A vibration testing 
program in a simulator should include a wide range of exciting frequen- 
cies to cover those found in the numerous types of military fixed-wing 
and helicopter aircraft. The program should also determine the effects 
of introducing the vibration excitation in several directions — normal to 
the lateral, longitudinal and vertical axes. 

To further approach a realistic simulation, the control sticks 
could be provided with a break-out force as found in actual aircraft. 

A trim switch could be included on the sticks to ascertain the problem 
this might produce with the rigid controllers. Finally, as a further 
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attempt to simulate the pilots' working condition in the cockpit, the 
test subject could be occupied with other duties during a run to evaluate 
the effects of distractions on his ability to use the various sticks. 

A variation of in-flight conditions such as different turbulence 
levels might have a large effect on pilot acceptance of a rigid stick. 
These different levels could be produced in the simulator by altering 
the speed, magnitude, and smoothness of the pip deflection on the 
oscilloscope display. 

Various flight modes should be studied to more accurately depict 
the applicability of the rigid control system. An example would be to 
find pilot performance and opinion using the four control sticks in a 
simulated landing approach. 

If investigations such as those mentioned still indicate the 
feasibility of a rigid cockpit control system in operational aircraft, a 
flight testing program should be Initiated for a definite and conclusive 
answer to the question. 
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APPENDIX A 



LIST OF EQUIPMENT UTILIZED 

1. Two Channel Electric Recorder 

Brush Electronics 

2. Low Frequency Function Generator, Model 202A 

H ew lett - Pa c ka rd 

3. Models I20A and 130A Oscilloscope 

Hewlett-Packard 

4. Universal Eput and Timer 

Berkeley Division Beckman's 

5. Pace TR-10 Analog Computer, Model 7350 

Electronic Associates, Inc. 

6. Lab-Chron 115 -volt 60 cps Timer 

Laboratory Industries , Inc . 

7. Power Supply (two) Model 3569 15/30 v. 

System Research Corporation 
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PILOT SCORING AND RATING DATA 
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: SCORING AND RATING DISTRIBUTIONS FOR PILOT CLASfES 
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